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(57) Abstract: In a set of local terminals (2-5) 
communicating via a network (6) to constitute a 
collaborative system (1), each terminal (2) receives 
data via the network and processes the data to 
present information via a user interface (9). A 
network performance monitor (11) determines a 
network performance parameter representative of 
performance status of the network in facilitating 
collaborative communication between the terminals. 
The network performance monitor outputs the 
network performance parameter to an output 
modifier (12) which controls the output generated 
by a user interface output generator (10) such that 
the information presented by the user interface to 
the user is varied according to the value of the 
network performance parameter for facilitating 
user perception of the performance status of the 
network. Where a scene is presented to the viewer, 
a perceptual metaphor may thereby be conveyed 
such as to associate fine sunny weather conditions 
with a good state of well-being of the network 
and to associate rainy or wintery conditions with 
a deteriorating state of well-being of the network. 
The process has application to virtual reality 
environments allowing interaction between virtual 
characters controlled by participants each provided 
with a respected one of the terminals at remote 
locations served by the network, as in the case of 
Internet based games. 
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NETWORK PERFORMANCE INDICATORS 

This invention relates to collaborative processes in 
which a group of terminals connected by a network operate 
in a manner which allows real time interaction between 
users of the terminals. The process may for example be 
one in which a virtual reality environment is created to 
allow .interaction between virtual characters controlled 
by participants each provided with a respective terminal 
at mutually remote locations served by the network. 
Such an environment may be used to- play a game, as a 
training and, as a means of communication or as part of 
a control system. Other examples include audio , video 
or audio-visual conferencing facilities in which 
participants are provided with audio-visual data streams 
from other participants in a communications session via 
networks such as the internet,. cellular mobile 
communications network, cable networks, telephone 
networks, or combinations thereof. 

The performance of networks such as the Internet used for 
such collaborative processes tends to vary according to 
prevailing conditions. For example, the time taken for 
data to traverse the network between a local terminal and 
a remote terminal will depend not only on the direction 
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of propagation but: on the current network conditions 
specifically affecting routes connecting these terminals* 
Other variables affecting quality of service of a network 
include the frequency with which packets of information 
are lost and need to be retransmitted and the overall 
throughput or rate of data traffic which is possible 
under prevailing conditions of network congestion. 

At each terminal r the user is provided with a user 
interface allowing the user to participate for example 
by receiving a display which is generated to represent 
information obtained by processing data received via the 
network. The behaviour of the collaborative system as 
perceived by the user should ideally be consistent. 
However r under conditions where network quality of 
service deteriorates, the user may be presented with 
unexpected artifacts in the manner in which information 
is presented, thereby detracting from the* ease with which 
the user may operate the terminal for interactive 
collaboration with the group. 

An aspect of the present invention provides a method of 
processing information in the terminal which allows the 
output provided to the user to be controlled or modified 
in a manner which is representative of at least one 
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aspect of network performance. In one embodiment, the 
user interface provides a display screen representing a 
scene representing a virtual environment and a measured 
network performance parameter is used as a control signal 
for controlling the manner in which the scene is 
rendered. A change in network quality is therefore 
accompanied by a modification in the manner in which the 
scene is rendered, the resulting modification being 
recognisable by the user so that the user is then able 
to modify the way in which the user interacts with the 
system via the user interface. For example, if the 
scene is represented in. a way which suggests to the user 
that communication with one of the entities of the 
collaborative group is subject to excessive latency, the 
user will allow additional time for that entity to react 
to any action taken by the user. 



In the case of a scene which is synthesised,. as in the 
case of a virtual environment, it may be appropriate to 
use a perceptual metaphor within the scene in order to 
convey to the viewer the state of well being of the 
network. One such perceptual metaphor is to associate 
fine, sunny weather conditions with a good state of well 
being and to associate rainy, stormy or wintery 
25 conditions with a deteriorating state of well being. 
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Another metaphor is to associate a bright sunny well lit 
scene with a good state of well being and to associate 
progressively darker and gloomier , poorly lit scenes with 
progressively deteriorating states of well being. 

In the case of a scene which is not synthesised but which 
is derived from a video stream obtained from a video 
camera of an actual subject, it may be appropriate to 
control the scene to represent a measured network 
performance parameter by varying the quality of 
reproduction of the video image, for example by allowing 
controlled break up of* the image, slowing the rate at 
which the scene is refreshed, reducing the size of. the 
image or partially obscuring part of the image as in a 
tunnel vision effect. These effects may of course also 
be applied to synthesised images if desired and may 
additionally be viewed as perceptual metaphors when used 
in an appropriate context. 

Embodiments of the invention will now be described by way 
of example only and with reference to the accompanying 
drawings of which: 

Figure 1 is a schematic diagram of a collaborative system 
comprising a group of terminals connected by a network; 
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Figures 2A to 2L illustrate examples of modifications to 
a scene to represent network conditions; 

Figure 3 is a schematic diagram showing greater detail 
of the network performance monitor and output modifier 
of the terminal of Figure 1; 

Figures 4 A to 4G are schematic diagrams of messages used 
in measuring network performance; 

Figure 5 is a schematic diagram illustrating operation 
of an actuator; 

Figure 6 is a schematic diagram illustrating hardware of 
the terminal of Figure 1; and 

Figure 7 is a representation of a method of evaluating 
network performance and presenting information to the 
user. 

Figure 1 illustrates a collaborative system 1 in which 
terminals 2, 3, 4 and 5 are connected by a network 6. 
In .this example, terminal 2 will be referred to as a 
local terminal and will be described in detail, it being 
understood that each of the remaining remote terminals 
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3, 4 and 5 has a corresponding structure and mode of 
operation . 

In this example, the network 6 consists of the Internet 
and the collaborative system 1 provides a distributed 
virtual environment in which the database for creating 
the virtual environment is distributed amongst the 
terminals 2,3,4 and 5. Users at each of the terminals 
2, 3, 4 and 5 control respective virtual characters which 
are represented in scenes for viewing the virtual 
environment in real time. 

In Figure 1, detail of the local terminal 2 includes a 
network interface 7 allowing two way communication with 
the network 6 and a data processor 8 which processes data 
received via the. network and via a user interface 9 to 
obtain information relating to the current state of a 
model of the virtual environment* A user interface 
output generator 10 provides an output to the user 
interface 9 for displaying the information as a scene 
representing the virtual environment as viewed by the 
character controlled by the local terminal 2. 

A network performance monitor 11 interacts with the 
remote terminals 3 r 4 and 5 via the network 6 to obtain 
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information on the status of the network, including the 
network conditions specifically affecting the manner in 
which each of the remote terminals- is currently able to 
interact with the local terminal 2. 

5 

The network performance monitor 11 provides an output 
modifier 12 with values of network performance parameters 
representing the network performance quality, of service 
and the output modifier interacts with the user interface • 
10 output generator 10 to control the output provided to the 

user interface 9 in a manner which is representative of 
the network performance parameters - 

Figure 2A illustrates schematically a display provided 
15 by user interface 9 representing three characters in a 

virtual environment under normal network conditions, the 
term "normal" here being used to signify a high quality 
of network service provided by network 6 . A first . 
character 20, represented as a triangle, is controlled 
20 by terminal 3 such that movement and actions made by the 

character follow instructions input via the user of 
terminal 3 and communicated as data via the network 6 to 
the remaining terminals 2, 4 arid 5. In terminal 2, the 
data is processed in data processor 8 and the character 
25 synthesised as part of graphics data representing a 
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scene. The user Interface output generator 10 receives 
the graphics data and generates signals for input to the 
user interface 9 which includes an audio visual display- 
apparatus . 

5 

Also shown in Figure 2A are a second character 21 
represented as a circle and controlled by the user of 
terminal 4 and a third character 22, represented as a 
square, which in the virtual environment is controlled 
10 by the user of terminal 5. 

If as an example the network conditions specifically 
affecting communication with terminal -3 become degraded 
such that data communicated between terminal 3 and the 

15 local terminal 2 becomes subject to delay or loss, or a 

combination thereof, to an extent which is likely to 
produce unexpected results or artifacts in the display 
output of the user, interface 9, the deterioration in 
network performance is detected by network performance 

20 monitor 11, enabling output modifier 12 to interact with 

the user interface output generator 10 in a manner which 
allows the reduced network performance to be indicated 
to the user* 

25 Since in this example the degradation in network 6 is 
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perceived by the local terminal 2 as being associated 
specifically with the network environment affecting 
terminal 3, it is appropriate to modify the manner in 
which the first character 20 is represented, without 
5 changing the manner in which the second and third 

characters 21 and 22 are represented to the user. 

Figure 2B illustrates one example of how such 
representation may be effected. In Figure 2B, a breakup 
10 technique is used to fragment the displayed 

representation of the first character 20 which is now 
represented as fragmented character 23. For example, 
breakup can be achieved by fragmenting the visual 
representation of a figure into a number of separate 
15 pieces. The user will identify this breakup in terms of 

understanding that the behaviour of the first character 
20 should be expected to change because of difficulties 
in communicating data with the remote terminal 3 , thereby 
enabling the behaviour of the user to be adapted to the 
20 prevailing pattern of response. The user may for example 

choose to take actions which interact with only the 
second and third characters 21 and 22 for so long as the 
first character 20 is- represented by fragmented characteb 
23. Once the networks status associated with terminal 
25 3 improves, detection of this improvement by the network 
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performance monitor 11 may result in the first character 
20 being restored to its unbroken status as shown in 
Figure 2A, following which the user can adapt his 
behaviour accordingly. 



The user may alternatively choose to continue to interact 
with the first character but will understand that while 
the fragmented character 23 is represented in the 
display , the response time associated with interactive 
behaviour will be degraded. 

In this manner the virtual reality environment may be 
made more user friendly since the user is able to 
perceive prevailing network conditions. 

Figure 2C illustrates an alternative representation for 
the example where the network conditions associated with 
remote terminal 3 are degraded such that the behaviour 
of the first character 20 is adversely affected. In this 
example, a probabilistic representation is used in which 
ghost images 24 and 25 are superimposed on the display 
at positions calculated to be possible positions of the 
first character 20. Where for example the first 
character 20 under normal network conditions is moving 
across the scene and subsequently the network conditions 



03097 194A2J_> 



WO 03/097194 



PCT/GB03/01135 



11 

affecting communication with terminal 3 are degraded ; it 
may be possible for the virtual environment model to 
predict further movement in the absence of incoming data 
based on previous behavioural patterns. Where a number 
of options are available for predicted movement, a 
different ghost image 24, 25 may thereby be generated to 
represent each of the possible options. Once the network 
conditions improve and data representing the control 
signals input by the user of terminal 3 are processed by 
the data processor 8, the normal representation of Figure 
2A may be resumed so that the ghost images disappear and 
the current intended position of the first character 20 
is represented to the user. 

The probabilistic representation of Figure 2C enables the 
user of terminal 2 to appreciate the degree of 
uncertainty associated with the intended position of the 
first character 20 in the virtual environment and enables 
the user to take appropriate action- In the absence of 
such a representation, the user could be presented with 
sudden unexplained erratic movement of the first 
character 20 following interruption and restoration of 
data communication via the network 6. The effect of the 
network performance monitor 11 and output modifier 12 is 
therefore to improve the. user's understanding and 
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appreciation of the way in which the characters of the 
virtual environment are being controlled and to enable 
the user to perceive network quality of service and its 
effects . 

In a further example, the network conditions affecting 
communication with the local terminal 2 are adversely 
affected so that all data communication from remaining 
terminals 3, 4 and 5 in the collaborative group are 
adversely affected in equal amount. Such degraded 
network conditions could be represented using the 
techniques of Figure 2B and 2C for example, applying the 
output modifier 12 to each of the 1 st , 2 nd . and 3 rd 
characters. Alternatively , a different representation 
indicating a global deterioration in network quality may 
be utilised,, the term "global" being here used to signify 
that the effect is common to data communications with all 
other terminals of the collaborative group/ typically 
arising from a degradation in network conditions 
localised to the local terminal 2. 

Figure 2D illustrates one example of how the existence 
of global .network deterioration may be represented to the 
user. In this example, the weather prevailing in the 
virtual environment is modified to provide a perceptual 
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metaphor of the prevailing network conditions. Since the 
user will associate fine sunny weather with good 
prevailing conditions and rain or snow with adverse 
conditions, it . is appropriate to represent adverse 
network conditions by precipitation in the scene 
representing the virtual environment. In Figure 2D, the 
first, second and third characters 20., 21 and 22 are 
observed through a rain pattern 26. The intensity of the 
rain pattern 26 may be varied to represent different 
degrees of adverse network . conditions . The use of 
weather control in the virtual environment is an example 
of a perceptual metaphor which relies upon the user's 
intuitive perception of adverse weather as an ominous 
portent in the virtual environment. 



The rain pattern 26 may be generated by the user 
interface output generator 10 in response to a command, 
from the output modifier 12 by applying a predetermined 
pattern to overlay the scene currently rendered as output 
20 to the user. 

in an alternative example in Figure 2E, the prevailing 
light intensity is varied as a perceptual metaphor 
representative of network conditions. Since the user 
25 will associate bright, sunny conditions with a conducive 
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environment, the overall light intensity level can be 
progressively darkened to metaphorically represent 
various degrees of adverse network conditions. Such 
darkening of the scene is intuitively perceived by the 
5 user as metaphorically signifying a degradation of 

network performance, allowing the user to adapt the 
manner in which he interacts with the virtual environment 
system accordingly. 

10 Figure 2E is therefore a further example of a 

representation of adverse global network conditions. It 
may be appropriate additionally to indicate network 
conditions associated with individual entities of the 
collaborative group, as previously discussed in relation 

15 to Figures 2B and 2C. Figure 2F is an example of 

representing global network conditions by reduced light 
level in combination with using the break up technique 
of Figure 2B to represent adverse network conditions 
which additionally specifically affect remote terminal 

20 3. 

Other forms of representation of global network 
conditions include the example of Figure 2G, 2H and 21. 
Figure 2G represents a marginal deterioration in global 
25 network conditions by a tunnel effect, labelled 
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"TUNNEL 1" in Figure 2G, in which a marginal region 27 
overlays the scene generated in the user interface output 
generator 10 to create a tunnel vision effect. A greater 
degree of network deterioration is signified by the 
"TUNNEL 2" of Figure 2H in which a larger marginal region 

28 obscures the periphery of the scene. Finally, in the 
"TUNNEL 3" of Figure 21 , an even larger marginal region 

29 obscures the periphery of the scene. The output 
modifier 12 commands the user interface output generator 
10 to apply TUNNEL 1, 2 or 3 effects according to the 
measure of network performance detected by the networks 
performance monitor 11. 

The degree of tunnel effect is appreciated intuitively 
by the user of terminal 2 as signifying progressive 
slowing or interruption of the processing underlying the 
virtual environment represented in the scene and 
therefore constitutes a further example of a perceptual 
metaphor . 

Similarly, Figures 2J, 2K and 2L illustrate a further 
example' in which the magnification applied to the 
rendered scene is varied to provide progressively 
decreasing magnifications 1, 2 and 3 respectively 
corresponding to progressively deteriorating global 
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network conditions. 

Figure 3 illustrates schematically an embodiment of the 
network performance monitor 11 for use in the 
collaborative system" 1 of Figure 1. The network 
performance monitor 11 interacts with the network 6 via 
the network interface 7 using probe messages to derive 
information used to update a number of entity state 
models 30 , each of which constitutes a model of network 
conditions prevailing at a ■ respective one of the 
terminals 3 f 4 and 5, each of which being regarded as a 
separate entity requiring a separate model. In the 
example of Figure 3, three entity state models 30a, 30b 
and 3 0c are illustrated. The number of entity state 
models is however variable according to the number of 
entities which need to be modelled and may for example 
vary within a given session as one or more terminals 
joins or leaves the collaborative group. 

Respective outputs from- each of the entity state models 
30 are provided to the output modifier 12 and provide the 
required data to enable the output modifier to interact 
with the user interface output generator 10 to modify the 
display as appropriate to prevailing network conditions , 
as described above in Figure 2 . 
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The entity state models 30 also interact with a global 
state model 31 which models the overall network 
conditions as perceived by the local terminal 2. In this 
way, the onset of network deterioration coinmon to all 
remote entities can be interpreted as being a global 
effect specifically associated with the local terminals 
An output from the global state model 31 to the output 
modifier 12 allows the output modifier to interact with 
the user interface output generator 10 to create global 
effects on the display as illustrated above for example 
in Figures 2D to 2L. 

The entity state models 30 exist as software modules 
controlled by a manager 32 which keeps track of the 
number of collaborative entities in the collaborative 
system 1 and creates or destroys entity state models 
accordingly. New entity state models 30 are created by 
a generator 33. The output modifier 12 similarly 
<*p&f>£SSe£ a series of modules 12a, 12b and 12c in respect 
each one of the entity state models 30, these modules 
being similarly managed by the manager 32, the output 
modifier 12 also including a global module 12 G for 
receiving signals from the global state model 31. 

A status provider 34 analyses information obtained from 
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probe messages received via the network interface 7 and 
inputs appropriate data to the entity state models 30. 

A probe generator 35 generates outgoing probe messages 
using a multicast communications protocol to send 
messages to each entity of the collaborative group , in 
this case the terminals 3, 4 and 5. As illustrated in 
Figure 3, the status provider 34 is provided with a loss 
detector 36 specifically for keeping track of packet loss 
in each direction. 

Each of the terminals 3, 4 and 5 similarly outputs probe 
messages for obtaining network status information. Local 
terminal 2 receives such probe messages and replies using 
responder 37 . 

The probe messages broadcast by local terminal 2 are 
therefore subject to similar reception and response by 
corresponding features of the terminals 3, 4 and 5. A 
set of standard message types is provided for 
transmission and reply between the entities of the group 
to enable the network performance monitor 11 to obtain 
necessary information. 

Figure 4A illustrates a probe message in schematic form 
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in which the data fields of the message will now be 
described in general terms- A first field TP represents 
the type of message , in this case a probe message. A 
next field IDS represents the identity of the source of 
the probe message, in this case the terminal 2. Field 
TSS is a time stamp for the source of the probe message, 
representing the time at which the probe message is 
broadcast. Field SNS represents the sequence number of 
the probe message, being the position of the probe 
message in a sequence of probe messages originating from 
the source identified by field IDS. 

Figure 4B illustrates a probe reply message which 
similarly includes a type field TP, a source ID field IDS 
and a source time stamp TSS . The request probe reply 
message is- issued by the responder 37 when a probe reply 
message is received. The request probe reply message 
also includes a response identification field IDR 
indicating the unique identification of the entity 
responding to the probe message and a response stamp time 
field TSR indicating the time stamp of when the response 
was sent. The message also includes a source sequence 
number field SNS indicating the sequence number of the 
probe message to which this probe reply message responds. 
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By sending a probe message and receiving a probe reply 
message from each terminal, the local terminal 2 is able 
therefore to determine from the time stamp fields TSS and 
TSR the time taken to traverse the network with the probe 
and probe reply messages. The source sequence numbers 
also enable the loss detector 36 to keep track of those 
probe messages which do not elicit a probe reply message 
thereby indicating that a packet loss has occurred. 



10 Figure 4C illustrates a request probe reply message which 

. is generated for interrogating a selected one of the 
terminals in the group when the status provider 34 has 
identified a particular problem in communicating with 
that terminal. The message includes type field TP and 

15 source ID field IDS as well as a time stamp field TSS, 

a source sequence number field SNS and a target 
identification field IDT. The field IDT identifies the 
terminal to which the message is directed and. from which 
a specific response is requested. 



Figure 4D illustrates an announce message which is 
generated when an entity joins a collaborative group and 
wishes to announce its inclusion in the system. The 
announced message comprises type field TP and announce 
identification field IDA indicating the unique 
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identification code of the newly joined entity. 

Figure 4E illustrates an announce reply message by which 
terminal 2 responds to the announced message of Figure 
5 4D, the message including TP and IDA fields as well as 

a reply identification field IDR indicating the identity 
of the terminal which is responding to the announcement. 

Figure 4F illustrates a leave message used to announce 
10 that an entity is leaving the collaborative group during 

the session and consists of type field TP and a leave 
identification field IDL indicating the unique 
identification code of the entity leaving the session. 

15 Figure 4G illustrates an alive message consisting of type 

field TP, source identification field IDS and source 
sequence number field SNS indicating the current sequence 
number of the message. This message is used to confirm 
the continued presence of an entity in the group and to 

20 detect packet loss. 

From the information received in the probe reply 
messages, the status provider 34 is able to calculate the 
time taken to reach each entity and the time taken for 
25 the return transmission. This information indicating 
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network latency local to each entity is then passed to 
the respective entity state model 30. The loss detector 
36 also derives information from the sequence number of 
probe reply messages concerning packet loss and similarly 
passes this information to the appropriate entity state 
model • 

For each instance of a response message not being 
received from one of the entities of the collaborative 
group , the status provider 34 initiates the output of a 
request probe reply message targeted specifically to the 
terminal from which no reply was received. Further 
information may then be derived from any response 
received to this specific request. 

Figure 5 illustrates schematically the manner in which 
the output modifier 12 responds to data provided by one 
of the entity state models 30 which in this example is 
model 30a. Variable parameters provided by the model 30a 
are input to an analyser 50 which compares the variables 
with threshold levels. A decision is taken in accordance 
with predetermined threshold criteria as to whether 
action is required to modify the output to the user 
interface 9. if for example the round trip time for 
probe transmission and return exceeds a threshold level 
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of 0.5 seconds, action is triggered whereby an actuator 
51 is controlled to interact with the user interface 
output generator 10. As an example, the user interface 
output generator 10 comprises a. graphics data model 52 
5 which creates graphics data for generating a scene 

describing a virtual environment, the data being 
processed by renderer 53 which creates output signals to 
a display 54. In this example, the actuator 51 interacts 
with the renderer to cause the breakup of the first 
10 character 20 in Figure 2A to result in the fragmented 

character 23 of Figure 2B, thereby indicating to the user 
viewing the display that network conditions local to the 
entity associated with terminal 3 has deteriorated to an 
extent whereby the integrity of the display of virtual 
15 environment is prejudiced. Presented with this breakup 

of only one of the characters in the virtual environment, 
the user intuitively appreciates that the display 
position and other characteristics in the environment 
associated with this entity should be regarded as being 
20 suspect or subject to erratic behaviour during a period 

in which the breakup continues to be shown. The user may 
therefore modify the manner in which he interacts with 
the collaborative group accordingly. 
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When a new entity joins the group, this is signalled by 
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the transmission from that entity of an announced message 
as illustrated in Figure 4D in which the IDA field 
declares the identity of the new entity- Similarly when 
an entity leaves the group , this is announced to the 
remaining entities by the leave message as illustrated 
in Figure 4F which includes the IDL field declaring the 
identity of the leaving entity. 

As described above, the manager 32 is responsive to such 
messages to ensure - that the correct number of entity 
state models 30 is provided in the operating environment 
of the network performance monitor "11 and corresponding 
analysers 50 and actuators 51 exist in the output 
modifier 12. 

The output modifier 12 may be provided with more than one 
threshold level for a given state model , for example 
having first, second and third thresholds against which 
a given network quality parameter such as latency is 
compared. Corresponding first , second and third levels 
of effect may thereby be applied by actuator 51 to 
provide correspondingly different levels of modification 
to the display as illustrated in Figure 2A or 2C. 

The selection of appropriate form of modification for a 
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given display may depend on the type of data which is 
being output for display. For example, both 2D and 3D 
graphics data are characterised by an alpha-channel which 
can be variably controlled to change the level of 
5 transparency of a constituent character in a scene. For 

2-D graphics data systems, it may be more convenient to 
paint peripheral tiles black, as for example in Figures 
2G, 2H and 21. 

10 For 3-D graphics data systems, it may be more convenient 

to adjust the intensity as a whole as in Figure 2E since 
is normal for such, data to define a light object in a 
scene and for the intensity of the light object to be 
variably controlled. 

15 

Other forms of modifier may be appropriate for systems 
such as video conference systems. An actuator could for 
example break up video composition of the entire scene 
to indicate global network problems or break up video 

20 composition within one of the number of windows within 

the display containing video from a particular entity 
where network problems specifically affecting only that 
entity have been detected. The video could alternatively 
be subject to freeze frame, or reduced frame size to 

25 ■ indicate network degradation. In an audio visual 
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conference system the audio signal could be modified for 
example by varying audio volume or applying an audio 
processing effect such as distortion , frequency shift or 
any other number of available special effects. Such 
modif ications are examples of qualitative modifications 
to the information delivered to the user via the user 
interface. In such audio visual conference systems, the 
qualitative modifications are used to make the user aware 
of the state of the network, in contrast to known systems 
in which various techniques are utilised to mask the 
state of the network, for example by compensating for 
degradation in the audio or visual channel. 

A number of perceptual metaphors or qualitative 
modifications may be combined or made selectively 
available. 

In the above described examples, the control of output 
generation compares a measure of network performance with 
a number of thresholds. Alternative embodiments are 
envisaged in which the control of output generation 
varies in a continuous manner which is not dependent upon 
any threshold so that for example a perceptual metaphor 
effect can be varied in intensity in a continuous manner 
which is determined by a continuously variable parameter 
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or a function of a number of parameters. Alternatively, 
control of the output may utilise a hybrid arrangement 
Which is partly triggered by one or more threshold and 
partly variable in a continuous manner over part of the 
range of variation of the effect. 

In preceding figures , the function of local terminal 2 
has been illustrated using a functional block 
representation. Figure 6 is an example of a hardware 
representation of a terminal for achieving the above 
described function. A processor 60 is connected via a 
bus 61 to network interface 7 and is also provided with 
read only memory 62 and random access memory 63. A 
reader 64 allows a storage medium 65 to input program and 
data code. User interface 10 is also accessible via bus 
61. 

Processor 60 may be operated to perform the method steps 
of the present invention in accordance with program code 
received via storage medium 65 or via a signal 
communicated via the interface 7, as in the case of 
program code downloaded over a network such as the 
internet. The program code provides instructions for 
controlling the processor 60 to perform steps of a method 
as summarised in Figure 7. Data from network 6 is 
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received at step 71 and processed at step 72 to obtain 
information which is to be presented to the user. The 
information may be in the form of a virtual environment 
or communications information, depending upon the 
application of the collaborative system. 

At step 73, output signals are generated for presenting 
the information via the user interface at step 74. 

Network performance is continuously measured at step 75 
and at step 76 the output generation step 73 is 
controlled, in dependence upon the information obtained 
from measuring network performance. Step 76 generally 
results in a modification -to the manner in which the 
output signals are generated in step 73 in order to 
indicate to the user the status of the network 6, either 
globally or specific to one of the collaborative 
entities, the modification generally resulting in a 
perceptual metaphor of a displayed scene or a qualitative 
change in the manner in which the information is 
presented via the user interface. 



Alternative embodiments are envisaged in which user 
interfaces other than visual displays are utilised and 
in which appropriate control of output generation is 
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applied. For example, a user interface may be a haptic 
interface for presenting stimuli via a touch response, 
and olfactory interface for presenting stimuli for the 
user's sense of smell, an audio interface or an interface 
5 for delivering stimuli for the user's sense of taste. 

Combinations of the above interfaces are also envisaged. 

The arrangement of Figure 5 may be modified such that 
multiple actuators exist for a single entity state model 
10 and analyser in order to apply multiple superimposed 

effects to the information displayed to the user. 

in the described specific embodiment, the collaborative 
group comprises a total of four entities. Entities may 

15 join or' leave the group thereby varying the number of 

entities so that there is no specific limit on the number 
of entities to which the embodiments of the present 
invention have application. Clearly however there must 
be at least one remote terminal in additional to the 

20 local terminal 2 for network performance to have 

significance. Embodiments are envisaged in which one or 
more of the terminals is a server providing information 
utilised by the terminals, as for example in the case of 
online games played via the Internet. 



25 



BNSDOCID: <WO 030971 94A2_I_> 



WO 03/097194 PCT/GBO3/01135 



10 



30 

The disclosed embodiments refer to network parameters of 
latency and loss. It is envisaged that other network 
parameters such as jitter (variance of delay) may be 
utilised for specific collaborative systems and the 
choice of which parameter (or combination of parameters) 
is appropriate will depend upon the nature of the network 
and on the nature of the collaborative task performed by 
the entities of the collaborative group. In some 
instances, it may be necessary or appropriate to display 
to the user more than one network parameter using 
separately identifiable means of modification to the user 
interface output. 
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CLAIMS: 

1. A method of operating a local terminal constituting 
one of a collaborative group of terminals comprising one 
or more remote terminals communicating with the local 
terminal via a network, the method comprising operating 
a computer system of the local terminal to perform the 
steps of: 

receiving data via the network from the remote 
terminals; 

processing the data to obtain information to be 

presented to a user; 

generating an output for presenting the information 

to the user via a user interface; 

determining at least one network performance 
parameter representative of a respective performance 
status of the network in facilitating collaborative 
communication between the terminals; and 

controlling the generation of the output in 
dependence upon the at least one network performance 
parameter such that the manner in which the information 
is presented to the user is variable according to the 
value of the network performance parameter for 
facilitating user perception of the performance status 
of the network. 
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2. A method as claimed in claim 1 wherein the 
determining step comprises determining respective network 
performance parameters relating to network conditions 
specifically effecting each remote terminal of the group 
of terminals. 

3. A method as claimed in claim 2 wherein the 
generating step generates an output representative of a 
plurality of objects wherein each object is 
representative of information obtained from a respective 
one of the remote terminals and wherein the generating 
step separately controls the manner in which the output 
for each object is generated according to the value of 
a respective network performance parameter specifically 
affecting the remote terminal associated with that 
ob j ect . 

4 . A method as claimed in any preceding claim wherein 
the determining step comprises determining at least one 
global network performance parameter which is 
representative of network quality of service affecting 
the local terminal and thereby affecting communication 
between the local terminal and all remote terminals of 
the group . 
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5. A method as claimed in claim 4 wherein the control 
step comprises controlling the generation of the output 
such that the manner in which the information is 
presented to the user is variable according to the global 
network performance parameter. 

6. A method as claimed in any preceding claim wherein 
the controlling step comprises controlling the generation 
of the output such that the variation in network 
performance parameter is presented using a perceptual 
metaphor . 

7. A method as claimed in any of claims 1 to 5 wherein 
the controlling step comprises controlling the generation 
of the output such that the quality of reproduction of 
the information presented to the user is variable 
according to the network performance parameter. 

8. A method as claimed in any preceding claim wherein 
the step of determining the network performance parameter 
comprises measuring network quality of service by 
transmitting and receiving messages via the network. 

9. A method as claimed in claim 8 further comprising 
the step of monitoring the number of terminals in the 
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collaborative group and the identities thereof. 

10. A method as claimed in claim 2 wherein the step of 
determining network perf ormance parameters comprises 

5 operating a respective model of network conditions for 

each terminal of the group. 

11. A method as claimed in claim 10 wherein the 
determining step further comprises managing the number 

10 of models to correspond in number and identity to 

respective ones of the terminals active in the group. 

12. A method as claimed in any preceding claim wherein 
the processing step generates graphics data and wherein 

15 the controlling step comprises controlling the manner in 

which the graphics data is rendered. 

13. A method as claimed in any preceding claim wherein 
the controlling step comprises comparing a network 

20 performance parameter with one or more thresholds and 

determining the manner in which the output is varied 
according to the result of comparison. 

14. A method as claimed in any preceding claim wherein 
25 the output is generated for displaying a scene to the 
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user via a user interface. 

15. A method as claimed in claim 14 wherein the output 
is controlled to vary the extent to which an object 
represented in the display is fragmented. 

16. A method as claimed in claim 14 wherein the output 
is controlled to add ghost images indicating possible 
positions of a object represented in a scene. 

17. A method as claimed in claim 14 wherein the output 
is controlled to vary the prevailing weather conditions 
represented in a scene. 

18. A method as claimed in claim. 14 wherein the output 
is controlled to vary the overall light level in a scene. 

19. A method as claimed in claim 14 wherein the output 
is controlled to vary a degree of restriction applied by 
a tunnel effect superimposed on the scene. 

20. A method as claimed in claim 14 wherein the output 
is controlled to vary a magnification applied to the 
scene. 
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21. A method as claimed in any preceding claim wherein 
the processing step comprises generating a distributed 
virtual environment in which data defining the state of 
the environment is distributed amongst the terminals of 
the group and is communicated therebetween via the 
network. 

22. A method as claimed in any of claims 1 to 20 wherein 
the processing step comprises playing an online game. 

23. A method as claimed in any of claims 1 to 20 wherein 
the processing step comprises processing video data to 
provide a video conference. 

24. Apparatus comprising a local terminal constituting 
one of a collaborative group of terminals comprising one 
or more remote terminals communicating with the local 
terminal via a network, the apparatus comprising a 
computer system of the local terminal and comprising: 

an interface for receiving data via the network from 
the remote terminals; 

processing means for processing the data to obtain 
information to be presented to a user; 

generating means for generating an output for 
presenting the information to the user via a user 
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interface; 

determining means for determining at least one 
network performance parameter representative of a 
respective performance status of the network in 
facilitating collaborative communication between the 

terminals ; and 

control means for controlling the generation of the 
output in dependence upon the at least one network 
performance parameter such that the manner in which the 
information is presented to the user is variable 
according to the value of the network performance 
parameter for facilitating user perception of the 
performance status of the network. 

25. Apparatus as claimed in claim 24 wherein the 
determining means comprises means for determining 
respective network performance parameters relating to 
.network conditions specifically effecting each remote 
terminal of the group of terminals. 

26. Apparatus as claimed in claim 25 wherein the 
generating means is operable to generate an output 
representative of a plurality of objects wherein each 
object is representative of information obtained from a 
respective one of the remote terminals and wherein the 
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generating step separately controls the manner in which 
the output for each object is generated according to the 
value of a respective network performance parameter 
specifically affecting the remote terminal associated 
with that object. 

27. Apparatus as claimed in any of claims 24 to 26 
wherein the determining means is operable to determine 
at least one global network performance parameter which 
is representative of network quality of service affecting 
the local terminal and thereby affecting communication 
between the local terminal and all remote terminals of 
the group. 

28. Apparatus as claimed in claim 27 wherein the control 
means is operable to control the generation of the output 
such that the manner in which the information is 
presented to the user is variable according to. the global 
network performance parameter. 

29. Apparatus as claimed in any of claims 24 to 28 
wherein the control means is operable to control the 
generation of the output such that the variation in 
network performance parameter is presented using a 
perceptual metaphor. 
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30. Apparatus claimed in any of claims 24 to 28 wherein 
the control means is operable to control the generation 
of the output such that the quality of reproduction of 
the information presented to the user is variable 
according to the network performance parameter. 

31. Apparatus as claimed in any of claims 24 to 30 
wherein the determining means comprises means for 
measuring network quality of service by transmitting and 
receiving messages via the network. 

32. Apparatus as claimed in claim 31 further comprising 
means for monitoring the number of terminals in the 
collaborative group and the identities thereof. 

33. Apparatus as claimed in claim 25 wherein the 
determining means comprises a respective model of network 
conditions for each terminal of the group. 

34. Apparatus as claimed in claim 33 wherein the 
determining means further comprises means for managing 
the number of models to correspond in number and identity 
to respective ones of the terminals active in the group. 
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Apparatus as claimed in any of claims 24 to 34 
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wherein an output of the processing means comprises 
graphics data and wherein the control means is operable 
to control the manner in which the graphics data is 
rendered . 

5 

36- Apparatus as claimed in any of claims 24 to 35 
wherein the control means comprises an analyser for 
comparing a network performance parameter with one or 
more thresholds and determining the manner in which the 
10 output is varied according to the result of comparison. 

37. Apparatus as claimed in any of claims 24 to 36 
wherein the output comprises signals for use in 
displaying a scene to the user via a user interface. 
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38. Apparatus as claimed in claim 37 wherein the control 
means is operable to control the output to vary the 
extent to which an object "represented in the display is 
fragmented. 

39. Apparatus as claimed in claim 37 wherein the control 
means is operable to control the output to add ghost 
images indicating possible positions of a object 
represented in a scene. 



25 
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40. Apparatus as claimed in claim 37 wherein the control 
means is operable to control the output to vary the 
prevailing weather conditions represented in a scene. 

41. Apparatus as claimed in claim 37 wherein the control 
means is operable to control the output to vary the 
overall light level in a scene. 

42. Apparatus as claimed in claim 37 wherein the control 
means is operable to control the output to vary a degree 
of restriction applied by a tunnel effect superimposed 
on the scene. 

43. Apparatus as claimed in claim 37 wherein the control 
means is operable to control the output to vary a 
magnification applied to the scene. 

44. Apparatus -as claimed in any of claims 24 to 43 
wherein the processing means is operable to generate a 
distributed virtual environment in which data defining 
the state of the environment is distributed amongst the 
terminals of the group and is communicated therebetween 
via the network. 

45. Apparatus as claimed in any of claims 24 to 43 
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wherein the processing means is operable for playing an 
online game. 

46. Apparatus as claimed in any of claims 24 to 43 
wherein the processing means to operable to process video 
data to provide a video conference. 

47. A storage medium storing processor implementable 
instructions for instructing a processor to carry out a 
method as claimed in any one of claims 1 to 23. 

48. A system comprising a collaborative group of 
terminals communicating* via a network, the system 
comprising: 

means for determining the performance status of the 
network in facilitating collaborative communication 
between the terminals ; and 

wherein at least one of the terminals comprises 
generating means for generating an output for presenting 
to a user information received via the network and 
control means for controlling the generation of the 
output in dependence upon at least one network 
performance parameter such that the manner in which the 
information is presented to the user is variable 
according to the network performance parameter for 
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facilitating user perception of the performance status 
of the network. 
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